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Figure 1: Stylized animation of a galloping horse. From left to right: line samples are extracted from a 3D model; active strokes track the 
samples; brush paths are attched to the strokes and stylized as circular arcs; two more frames of animation exhibit temporal coherence. 


Abstract 

This paper presents a method for creating coherently animated line 
drawings that include strong abstraction and stylization effects. 
These effects are achieved with active strokes : 2D contours that 
approximate and track the lines of an animated 3D scene. Active 
strokes perform two functions: they connect and smooth unorga¬ 
nized line samples, and they carry coherent parameterization to sup¬ 
port stylized rendering. Line samples are approximated and tracked 
using active contours (“snakes”) that automatically update their ar- 
rangment and topology to match the animation. Parameterization 
is maintained by brush paths that follow the snakes but are inde¬ 
pendent, permitting substantial shape abstraction without compro¬ 
mising fidelity in tracking. This approach renders complex models 
in a wide range of styles at interactive rates, making it suitable for 
applications like games and interactive illustrations. 

Keywords: NPR, line drawings, temporal coherence, snakes 

1 Introduction 

Line drawings created by artists often include stylistic effects such 
as textured brushes and graceful, curved strokes. These effects re¬ 
flect the motion of an artist’s tool, and as such are fundamentally 
2D in appearance. The main challenge in recreating such effects 
for computer animation is to create lines that smoothly track the 
3D scene while maintaining the desired 2D texture and shape. This 
paper presents a method to create such a set of temporally coher¬ 
ent, stylized lines from an animated 3D model. We term these 
lines active strokes , because they adapt and extend active contours 
(“snakes”) [Kass et al. 1988] to stylized rendering. Active strokes 
automatically update their topology and shape as the underlying 3D 
scene animates, and provide coherent parameterization for texture 
and shape abstraction (Figure 1). Our approach performs at inter¬ 
active framerates, making it suitable for artistic exploration, games, 
interactive visualizations and illustrations. 


The definitive version is available at http://diglib.eg.org/ 


Stylization effects such as texture and shape abstraction require that 
lines be represented as connected curves. Suitable curves can some¬ 
times be extracted and stylized by traversing the 3D scene in object 
space [Northrup and Markosian 2000; Grabli et al. 2010]. In order 
to produce smooth, coherent animation, several approaches have 
been proposed to preserve the parameterization of object-space 
lines over time [Kalnins et al. 2003; Benard et al. 2010; Buchholz 
et al. 2011; Karsch and Hart 2011]. However, for models of moder¬ 
ate complexity like the Stanford bunny shown in Figure 2, object- 
space line extraction may produce many noisy, short segments that 
confound stylization with spatial and temporal coherence. More¬ 
over, object space lines lack natural level of detail (LoD) control, 
and offer no straightforward mechanism to depart from the under¬ 
lying surface in support of 2D shape abstraction effects. 


Active strokes avoid these issues by operating in image space, us¬ 
ing line samples output by filter-based line extractors (e.g. [Saito 
and Takahashi 1990; Raskar and Cohen 1999; Lee et al. 2007]). 
Since no natural connectivity exists between these samples, active 
strokes first vectorize the line samples using snakes. Like tradi¬ 
tional snakes, our snakes conform to the changing shape of the fea¬ 
tures in the scene. Unlike traditional snakes, they also need to adapt 
their topology and overall arrangement during animation. As the 
snakes split and merge to track the line samples, their parameteri¬ 
zation may change discontinuously. Active strokes therefore carry 
independently parameterized brush paths on top of the snakes to 
maintain consistent stretches of parameterization from one frame 
to the next (Figure 3). Since the snake handles tracking of mo¬ 
tion in the scene, the brush paths may deviate substantially from the 
position of the underlying line to support shape abstraction. 


The key innovation of our method is the introduction of an indepen¬ 
dent 2D entity, the active stroke, that approximates lines extracted 
from the 3D model and persists from frame to frame. We present 
new mechanisms to add, remove, trim, extend, split and merge these 
strokes to match animated feature lines. Our method provides level 
of detail control, operates on line samples from any type of line ex¬ 
tractor, and supports new stylization effects such as shape abstrac¬ 
tion for lines. We demonstrate our method with a line rendering 
system that achieves interactive frame rates and provides a broad 
range of styles. 





(a) silhouettes in 3D (b) side view of (a) (c) snakes 


Figure 2: Silhouette coherence. Frame 1: (a) Silhouettes in object 
space appear to be continuous in the image, but (b) are extracted 
as many disconnected parts, (c) Snakes recover 2D connectivity. 
Frame 2: (a) The silhouette is visually coherent with frame 1, but is 
(b) composed of a different set of disconnected parts. Nevertheless, 
the snakes (c) provide coherence. 


2 Related Work 

Line stylization. Image processing is the simplest, most efficient 
way to extract lines from 3D models. The seminal work of Saito 
and Takahashi [1990] filters depth and normal maps to extract 
contours and creases. It produces line drawings with natural 
coherence and LOD in 2D, but does not support much stylization 
or abstraction due to the raster nature of the output. Generalizing 
the method of Lee et al. [2007], Vergne et al. [2011] combine 
2D local fitting with spatially-varying convolution to produce 
more complex styles. But their implicit definition is without line 
parameterization, preventing the application of stroke texture or 
abstraction. Our approach supports image-based line extraction, 
but recovers connectivity and parameterization in support of such 
effects. 


Object space methods extract connected paths on the surface of the 
model. These paths have obvious parameterization for each individ¬ 
ual frame [Grabli et al. 2010], but the parameterization can change 
abruptly during animation due to changes in line topology. Kalnins 
et al. [2003] proposed the first complete method to enforce coher¬ 
ent parameterization by using the connectivity of smooth silhou¬ 
ettes [Hertzmann and Zorin 2000] to propagate stylization. This 
approach works well for simple models with few lines, where the 
mapping between object and image space is almost one-to-one. 
Models of moderate complexity like the Stanford bunny, however, 
produce many, tiny line fragments that collide in image space, 
thwarting effective propagation (Figure 2). The method of Benard 
et al. [2010] enforces a common parameterization for nearby lines 
(thereby making short segments behave as parts of a longer line) but 
this solution is generally suitable only for nearby lines that are also 
parallel. The method of Karsch and Hart [2011] tracks snake-like 
snaxels on the 3D object rather than re-extracting at each frame, 
avoiding propagation of parameterization. Because the snaxels live 
in object-space, however, they cannot easily provide LoD control 
or abstraction, and do not appear to address the aforementioned line 
fragmentation problem. 


Another contribution of Kalnins et al. is the use of multiple brush 
paths per silhouette. The parameterization of each brush path is in¬ 
dependently optimized to find a compromise between uniformity in 
screen-space and in object space. Our brush paths behave similarly, 
with a few differences. Their brush paths both track 3D motion and 
transmit parameterization from frame to frame. Our snakes handle 
the former role, so that our brush paths can be abstracted, deviat¬ 
ing far from the snakes without compromising the quality of track¬ 
ing (Figure 3). In addition, we employ the self-similar line artmap 
(SLAM) textures of Benard et al. [2010] to allow extra flexibility in 
the brush path parameterization. 

To date most line stylization approaches operate in an online fash¬ 
ion, i.e., without knowledge of the future positions of the lines. 
Buchholz, et al. [2011] present a method for parameterizing lines 
across an entire animation, allowing lines to anticipate topology 
changes and merge smoothly. While presented in the context of 
object-space occluding contours, thus sharing the difficulties of 
other object-space approaches, the spatio-temporal concept is in¬ 
dependent and could be applied to our active strokes, though we 
have not explored that option here. 


Active contours in NPR. Introduced by Kass et al. [1988], 
snakes are widely used in computer vision and medical imaging 
applications because of their robustness to noise. They have also 
been applied in several NPR contexts: line extraction [Karsch and 
Hart 2011], painterly rendering [Hertzmann 2001], and video styl¬ 
ization [Agarwala 2002; Agarwala et al. 2004]. As mentioned, 
the snaxels approach of Karsch and Hart works in object space, 
where the topology of the contours generators forms a set of closed 
loops. On the other hand, methods that use snakes to delineate im¬ 
age space region boundaries need to deal with a network of open 
curves, and thereby take on the need to deal with more complex 
topology changes over time. The systems of Agarwala et al. seg¬ 
ment and track color regions in videos with B-spline snakes for co¬ 
herent stylization. The user manually draws the initial outlines of 
the contours and the outlines are propagated to track features in the 
video. These systems were designed for processing video offline 
and can therefore rely on heavy-weight optimization and manual 
correction. In contrast, our approach is designed to track motion 
field in animated 3D scenes where the camera path is not known in 
advance, for example games. Therefore it must be interactive, fully 
automatic, and cannot rely on “future” frames. On the other hand, 
we benefit from the quality of our input motion - the motion field 
extracted from 3D is generally less noisy than optical flow from 
video. 


Figure 3: Each brush path (dotted) is defined relative to a snake 
(solid), but can deviate substantially to support abstraction effects 
as shown in Figure 1. As the knot turns, the snake topology evolves; 
for example the orange snake merges with the purple snake (top). 
Coherent arc-length parameterization is preserved by maintaining 
the beige brush path over the purple snake. 
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Figure 4: Stages of our line drawing pipeline, an overview of which is given in Section 3 below. 


Line drawing LOD. Line level of detail (LoD) control is nec¬ 
essary to produce clean, abstract drawings from complex models. 
Image-space approaches provide a natural limit on line complex¬ 
ity (image pixels), but it is often desirable to further reduce density. 
Object-space lines allow arbitrary line density, but provide opportu¬ 
nities for simplification both during stylization [Grabli et al. 2004; 
Cole et al. 2006] and as a post-process [Barla et al. 2005; Shesh and 
Chen 2008]. A common approach is to extract lines, measure their 
density in screen space, then either omit or merge lines in overly 
dense areas. Grabli et al. [2004] and Barla et al. [2005] found that 
good results could be achieved by removing only overlapping lines 
that also align in orientation. We adopt a similar, simple definition 
of the line coverage that allows efficient processing and explicit 
control over spatial and directional proximity. Finally, by generat¬ 
ing brush paths that are independent of the feature lines, we can eas¬ 
ily merge lines rather than simply omitting them, unlike approaches 
such as Cole et al. [2006]. 

3 Overview 

The key idea of our approach is create an independent set of curves 
(active strokes ) that track and approximate unorganized feature line 
samples and provide a coherent parameterization for stylization. 
Active strokes use snakes in order to track and vectorize the in¬ 
put feature samples. On top of the snakes an active stroke carries 
a set of brush paths that propagate coherent parameterization and 
give the final shape of the strokes. Figure 4 illustrates our pipeline. 

Our method takes as input a set of feature samples with 2D posi¬ 
tion and 2D tangent, and for animated scenes, 2D velocity. These 
samples can be extracted by any method that provides an orienta¬ 
tion at each pixel (e.g. [Lee et al. 2007; Vergne et al. 2011]), but 
the smoother and more coherent the input samples, the better the 
result. We have found that lines extracted with steerable quadrature 
pair filters [Freeman and Adelson 1991] provide especially good 
input. We describe our application of these filters in Appendix A. 

For the first frame only, we construct a set of snakes conforming 
to these feature samples via a stochastic process described in Sec¬ 
tion 5. 

For successive frames, the snakes track and vectorize the input sam¬ 
ples. To obtain a good tracking, we maintain two properties for 
our snakes: coherence , meaning snakes should evolve continuously 
from frame to frame as they track the features of the model; and ac¬ 
curacy , meaning snakes should faithfully represent the shape of the 
underlying linear features extracted from the scene. For vectoriza- 
tion, we maintain the following properties: coverage , each feature 
line should be covered by one snake and each snake should cover 
a feature; simplicity , snakes should maintain simple, smooth and 
clean topology; length , snakes should be as long as possible while 
respecting the previous criteria. 

To achieve coherence , snakes from the previous frame are advected 
to follow the motion field in the scene (Section 4.1). Next the 
snakes are relaxed to conform to the features in the current frame 
(Section 4.2) in support of accuracy. After advection and relax¬ 
ation, the goals of coverage, simplicity and length will generally 
have been violated. The ideal solution would involve a complex 


non-linear optimization, difficult to achieve in a system designed 
for interactive frame rates. Instead, we adopt a greedy approach and 
update each snake individually, using heuristics to move towards a 
better solution (Section 5). We obtain efficient vectorization by be¬ 
ginning with a good guess adapted from the previous frame. 

Finally, we build on top of the snakes a set of brush paths that pro¬ 
vide a coherent parameterization across frames (Section 6.2). By 
relying on the tracking behavior of the snakes, the brush paths can 
deviate widely from the underlying feature lines to achieve styliza¬ 
tion and abstraction goals (Section 6.1). The brush paths are then 
textured and drawn to produce the final illustration. 

4 Tracking Feature Lines 

The first step of our approach is to track the feature lines available 
in each frame of the animation using snakes: 2D polylines that act 
like elastic strands, simultaneously attempting to satisfy external 
attraction forces and internal smoothing forces. In our approach the 
snakes are attracted to feature lines and track them from frame to 
frame. At each frame, we move the snakes according to the motion 
field in the scene (Section 4.1) and then relax them to conform to 
the feature lines in the current frame (Section 4.2). 

4.1 Advection 

At initialization, each snake vertex is associated with a 2D feature 
sample (Section 5). During advection, the snake vertex is moved 
according to the projected 3D motion field of the scene. The 3D 
motion field can be computed either by back projecting the 2D sam¬ 
ples in the camera at frame i — 1, reprojecting these 3D positions 
into the camera at frame i and computing the difference or by using 
an optical flow computation when the geometry is not available. 

After advection, lines from frame i — 1 are nearby (but not per¬ 
fectly aligned with) features in frame i. At this stage, the method 
of Kalnins et al. [2003] performs an explicit search to associate line 
samples with features in frame i. Instead, we use relaxation to at¬ 
tach the advected snakes to features in frame i , as described next. 

4.2 Relaxation 

As described by Kass et al. [1988], snakes are parametric 2D curves 
v(s) = (x(s),y(s )), s £ [0,1] that minimize the energy E, de¬ 
fined as the sum of smoothing and data terms Ei nt and E ext : 

E = [ E int (v(s)) + E ext (y{s)) ds 
Jo 

The internal energy Ei nt ensures continuity (first-order membrane 
term weighted by a) and smoothness (expressed as second-order 
thin-plate term weighted by /3): 

Eint(v(s)) = ^(a(s)|v'(s)| 2 +/3(s)|v"(s)| 2 ) 

with v' and v" the first and second derivatives. In practice, we use 
constant a and [3 parameters (both equal to 0.001 in our experi¬ 
ments). 

































Addressing the accuracy goal, the external energy E ext attracts the 
snake to areas of interest - in our case, feature lines that are pro¬ 
jected and rasterized into a binary image /. This image is filtered 
by a Gaussian kernel whose gradient is the attraction field: 

Eext{v(s)) = -\ V [Gcr(v(s)) * J(v(s))] | 2 

The size a of the filter kernel controls the width of the attraction 
field around the contour. A larger filter makes relaxation more ro¬ 
bust to errors in advection, at the price of accuracy in tracking. In 
our experiments we use a = 8px for screen-resolution images. 

Following the approach of Kass et al. we optimize the energy E by 
gradient descent. The continuous snake v(s) is discretized as n ver¬ 
tices Vi and the energy is iteratively minimized by a semi-implicit 
Euler scheme. At each frame, a matrix of internal forces is con¬ 
structed and applied iteratively, updating the contours positions and 
their associated external forces at each iteration. This scheme is 
simple and fast so long as the advection step provides a good start¬ 
ing condition. But when advection fails to produce a good initial 
guess it is likely due to extreme motion of the model. In this case, 
temporal coherence may not be necessary or even expected. 

To maintain good sampling, especially while zooming, the snake 
resolution is updated after several iterations of relaxation. Via the 
approach of Delingette et al. [2000] we ensure an almost uniform 
sampling of the curves, with a maximum distance of (5 max = 6px. 

Snakes have a natural tendency to shrink like a rubber-band due 
to their internal forces. This shrinking tends to produce inaccu¬ 
rate results where snakes round off corners or fail to reach line 
endpoints. To overcome this behavior, we apply spring forces 
along each snake edge so that they maintain almost constant 
length if no tangential external force 
is applied. The spring forces are im¬ 
plemented as an extra term in the ex¬ 
ternal energy, similar to the approach 
of Mclnerney et al. [1995]: 



usually by relying on user input. Starting from this initial connec¬ 
tivity, some approaches have been proposed to modify topology of 
the contours during segmentation tasks [Mclnerney and Terzopou- 
los 2000; Delingette and Montagnat 2000; Ji and Yan 2002], We 
take inspiration from these methods to define a set of heuristic rules 
that allow the snakes to change their topology and overall arrange¬ 
ment during animation so as to match the underlying feature lines. 

We organize these heuristics as six operators - delete, split, trim, 
extend, merge, insert - which are applied sequentially in a greedy 
fashion on the vertices of the snakes, until they can no longer mod¬ 
ify the snakes. The order in which the operators are applied is im¬ 
portant. Deletion and splitting occur first, so as to allow subsequent 
trimming to produce clean junctions. Extension also benefits from 
following splitting, and must precede merging to allow snakes to 
join across a gap. Insertion of new snakes occurs to address fea¬ 
tures without coverage after all the other operations have had the 
opportunity to adjust the snakes (especially extension). So in sum¬ 
mary, we delete as much as possible, trim as much as possible, and 
so on; and finally insert new snakes where extra coverage is needed. 

On the other hand, the order in which the snakes are processed for 
each operation has relatively less influence on the quality of the final 
result. Nevertheless, we process snakes in order from longest to 
shortest, except for the trim operator for which we use the opposite 
order, with the goal of maximizing the length of the longer snakes. 


5.1 Operations 


Our goal is to cover each feature line with exactly one snake. Recall 
that each feature line is actually composed of an unorganized set of 
feature samples S that are typically sparse in the image. We define 
coverage to mean that a feature sample Si with tangent T t has a 
snake vertex Vj nearby whose local orientation tj is similar: 


covered (s^ = 3vj 


11 Si Vj\\ Tcover 

\Ti ■ tj | > 9 C 


/i-t-i+i =k(Li- \\v i+ i -«i ||) M ^ +1 ^ii 

^11 

and the symmetric case for fi-n -1 where k is the stiffness param¬ 
eter of the spring {k — 1 by default) and Li is the rest-length equal 
to \\v i+1 — Vi\\ at the previous frame. 


The constants r COV er and 0 C control, respectively, the apparent line 
density and the tendency for snakes to match the orientation of the 
underlying features. A related notion is footprint^) which de¬ 
scribes the set of feature points covered by Vj : 

footprint^) = (s, € S | (||s» - Vj\\ < r coyer ) A (| T t ■ g\ > 9 C )) 


5 Vectorizing Feature Lines 

During animation snakes must be created, deleted, or modified to 
preserve a good approximation of the underlying feature lines. Con¬ 
structing snakes from the feature samples is equivalent to vectoriz¬ 
ing line art. This is a difficult problem even for still images, and 
our scenario also requires that the vectorization be temporally co¬ 
herent. Fortunately, we can exploit the smooth tracking behavior of 
the snakes to ease the vectorization problem. Snakes from the pre¬ 
vious frame that have been advected and relaxed provide an excel¬ 
lent starting point for vectorizing the samples for the current frame, 
because the features they track are coherent in image space. 

However, starting with these snakes, we still need to address two 
challenges. The first challenge is that after advection and relaxation 
we generally have not met our unit coverage goal. Several snakes 
may now overlap in image space where a single snake would suf¬ 
fice. Also, new lines may have appeared in regions where no snake 
existed before, or conversely a snake may remain where feature 
lines no longer exist. The second challenge is snake topology. Un¬ 
like our scenario, typical formulations for parametric snakes need 
not address the issue of automatically inferring their connectivity, 


Initialization. For the first frame of an animation we start by cre¬ 
ating new snakes as follows. We randomly pick an uncovered fea¬ 
ture sample from which we grow a snake as far as possible using 
the extension operator described below. This process is repeated 
until it can no longer improve coverage. Though not providing op¬ 
timality guarantees, this approach is simple and fast, and it offers a 
reasonable first guess for subsequent processing. 

Delete. Snake vertices that are not covering any feature sample 
are deleted. This mechanism may remove an entire snake where a 
feature has become occluded, or may cut away portions of a snake 
in case of partial occlusion. 

Split. (Figure 5a) To accurately track sharp features of a 3D 
model, a snake is split at vertex Vi if a sharp angle appears there, 
found by: __ > 

Csplit = ti— l • ti -(-1 < 0 _|_ 

where ti -1 and U+i are the tangents at two neighboring vertices 
and 6± is a threshold constant for how much the snake is allowed 
to bend at a vertex. 






(a) Split 


(b) Merge 


Figure 5: Split and merge operations modify snake connectivity. 

(a) When a snake bends too sharply around a corner it is split. 

(b) When two snakes abut smoothly they are joined. 

Trim. (Figure 6a) Having deleted portions of snakes that do not 
cover features, and split at sharp corners, we next trim snakes in re¬ 
gions where parallel lines provide more coverage than needed. Sev¬ 
eral strategies might be possible, for example attempting to merge 
lines where they nearly overlap. However, we have found a simple 
policy that trims lines at their endpoints to be effective for achiev¬ 
ing unit coverage. Specifically, the three criteria for removing the 
endpoint Vi from a snake are that it is too close to a vertex Vj of 
a different snake, they are close to parallel, and that by removing 
Vi we will not leave a feature uncovered that would otherwise be 
covered by vp 


Ctrim 


||Vi - VjJI 

< 

T* trim 

\ti • tj 1 

> 

d // 

footprint(ui) 

C 

Ufc^ifootprint(u/ c ) 


where U and t 3 are their respective tangents, and r tr \m and 6// are 
threshold constants for how close and parallel these parts of the 
snakes may become before trimming occurs. 



(a) Trim 


(b) Extend 


Figure 6: Modifying snakes to improve coverage, (a) The endpoint 
of a snake is trimmed if it is too close and parallel to another snake. 
When multiple snakes overlap in screen space this operation moves 
towards unit coverage, (b) The end of a snake is extended when do¬ 
ing so adds coverage to otherwise uncovered features. In concert, 
the trim and extend operations tend to clean up T-junctions. 


where the two terms in parentheses have been normalized to the 
range [0,1] and a is a parameter that balances between them. 


Merge. (Figure 5b) To simplify the snake topology, two snakes 
are merged if they are locally parallel and their endpoints are close: 


Cmerge — 


n-tj > Op 

Ej • U > O/i 

\\Vj Vi\\ < Emerge 


These criteria may lead to multiple conflicting candidate mergers, 
for instance where three endpoints meet in a Y shape. Therefore, 
in a fashion similar to the extension process described above, we 
assign a goodness score that favors straightness and closeness: 


Emerge — ' 


min(ff • tj ,Tj • ti) — 0 // 
1 - On 


+ (!-«) 


! - \\V4 - Vi 


Extend. (Figure 6b) Next we extend snakes at every endpoint 
where doing so will offer coverage for otherwise uncovered fea¬ 
ture points whose tangents align locally with the snake. Specifi¬ 
cally, beyond the current endpoint Vi we will consider appending a 
new endpoint Vj at the location of every uncovered feature sample, 
based on four criteria: 


Cextend 


r -i covered (yf) 

\\Vj-Vi\\ < rextend 

u -JT > d// 

- > 9// 


where Vi and Vj are the endpoints of two distinct contours, U and tj 
their tangents, and fj and fj are tangents of hypothetical merger 
segments, shown in Figure 5b. We perform as many non-conflicting 
mergers as possible, in order of descending score. 

Insert. This final operation works like the initialization step de¬ 
scribed above, except that it occurs after all the other operations. Its 
role is to insert new snakes where new features have appeared that 
have not been addressed by, for example, extending existing snakes. 

5.2 Practical considerations 


where t % and t 3 are the tangents at the snake vertices, T 3 is the 
tangent of the feature sample, and r ex tend is a search radius. 

In general there may be many candidate feature locations for which 
the extension criteria above are satisfied. Therefore, we compute a 
goodness score g 3 that describes how well each candidate extends 
the snake and then choose the candidate with the highest score. The 
score favors aligning the tangents and a relatively short extension, 
as follows: 

min(tt-t7, \tj-Tj\) ^ / rextend - I \vj 

1 @ // J \ r extend r cover / 


Threshold Constants. The scope of action of the four operations 
that trim, extend, split and merge the snakes are given by thresh¬ 
old constants for various radii and angles. Several relationships 
between them are important. In order for the snakes to establish 
proper coverage between samples: r COV e r > <5 max /2. In order to 
permit extension to occur: rextend > r CO ver. In order to enforce 
hysteresis between split and merge events: 0± < 6//. Other con¬ 
stants trade off between accuracy and performance. In our experi¬ 
ments we have found the values effective for the full range of results 
shown: r CO ve r = 10px; r tr im = 5px; r ex tend = 20px; r merg e = 20px; 
0 C = 0// = 0.85; 9± = 0.5; a = 0.8. 


fl'extend — & 





















Acceleration structure. During advection and coverage up¬ 
dates, we often need to access the neighbors of snakes vertices or 
feature samples in the coverage radius. To expedite local search, 
we use the data structure of Delingette et al. [2000]: a randomized 
sparse regular grid (implemented by a hash table) containing all 
the snakes vertices and feature samples. With a grid cell size of 2 
^cover, we enumerate all points in a disc of radius r COV er by walking 
four cells. 

6 Brush paths shape and parameterization 

Snakes approximate and track the underlying feature lines from 
frame to frame. Based on this temporally coherent layer, we build 
a set of brush paths that are in charge of (1) providing the shape 
and position of the final strokes, and (2) of computing a temporally 
coherent parameterization. 

6.1 Brush path geometry 

We represent a brush path as a 2D polyline with vertices associ¬ 
ated 1 — 1 to a sequence of vertices on a single snake (Figure 3). 
The position of brush path vertices are described relative to their 
associated snake vertices to take advantage of the snake coherence 
during motion. The user controls a target length l max (potentially 
infinite) for the brush path allowing for short brush paths if desired. 
In addition, brush paths can have “overshoot” that extends their end 
segments. 

Smoothing in time During rapid camera movements, feature 
lines may appear or disappear abruptly. As snakes persist across 
frames, we can smooth out these events by assigning a lifespan to 
the snake vertices, as follows. Each snake vertex has a lifespan k, 
initially null. At each frame, if a vertex persists, its lifespan in¬ 
creases: n = max(ft + 1, K m ax) with ftmax a constant defined by 
the user (typically 3 in our experiments). Conversely, if a vertex 
disappears, its lifespan decreases (k = k — 1). This lifespan is 
transmitted to the brush path vertices and can be mapped to dif¬ 
ferent style attributes, such as line opacity or thickness. That way, 
brush strokes fade in and out progressively during K max frames. 

Shape Abstraction The shape of the brush path can deviate sub¬ 
stantially from the shape of its corresponding snake, which allows 
highly abstract styles. As a proof of concept we show two extreme 
examples that have never been animated before: straight brush 
paths and circular arcs (Figure 7). 

Drawing with straight brush paths implies to partition each snake in 
a set of approximately linear segments. We compute this partition 
using dynamic programming, similarly to the approach of McCrae 
and Singh [2009] for fitting sketched clothoid curves. 

Between two frames, the initially straight brush paths are propa¬ 
gated following the motion of the underlying snakes. However the 
shape of the snakes is likely to have changed. Consequently, the 
brush paths may not be straight anymore, and we may even need to 
break a straight path into two or more segments to better match the 
new shape. Such a decision is taken using the same dynamic pro¬ 
gramming approach applied independently on each brush path. In 
addition, wherever the policy described in Section 6.2 might con¬ 
sider merging brush paths, we ensure that the dynamic program¬ 
ming solution would not immediately split the merged path. 

Conveniently, the abstraction algorithm for straight brush paths is 
easily adapted for circular arcs. We use the “modified least squares” 
method of Umbach and Jones [2003] to determine a best arc and 
compute the fitting error for a sequence of vertices. 


Slowing motion When using strong abstraction, temporal arti¬ 
facts may still occur. For example a circular arc can suddenly 
switch from concave to convex, or a single segment can break into 
two. To soften these transitions, we provide the following optional 
smoothing mechanism. Suppose the smoothed position of a path 
vertex at frame i is s*. At frame i + 1, rather than sending the ver¬ 
tex to its target position U+i, we move it a step in that direction 
d — 1 Si): 

Si +1 = Si + min(l, d ma x/|| d ||) d 

where d max specifies a maximum step size. Note that Si and U are 
represented as offsets relative to the corresponding snake vertex and 
not absolute positions. Thus our mechanism leaves unchanged the 
motion of the model while smoothing the abstracted paths. The 
smoothing is applied on the endpoints of segments, whereas every 
arc vertices are processed (meaning that the latter shape deviates 
from exact circles during smoothed transitions). 

6.2 Brush path parameterization 

We need to parameterize these paths with temporal coherence dur¬ 
ing animation. Snakes track features smoothly evolving in 3D. In 
absence of visibility events, their intrinsic parameterization cor¬ 
relates with the 3D motion of the feature lines. But it does not 
correspond to screen-space arc-length, which induces compression 
artifacts. Moreover, occlusions and disocclusions produce sliding 
and popping. To avoid such effects, we recompute the parameter¬ 
ization at each frame based on the previous one, and we use the 
self-similar line artmap (SLAM) textures of Benard et al. [2010] to 
avoid compression when zooming. To handle topological events - 
merging, typically - we allow multiple brush paths per snake, and 
forbid merging of two brush paths with different parameterization. 

Parameterization fitting. Similarly to the 2D coherence strat¬ 
egy of Kalnins et al. [2003], we assume that a brush path repre¬ 
sents the interaction between a drawing tool and 2D paper. Thus 
it should be uniform in screen-space, implying that its parameter¬ 
ization T is linear with respect to arc-length s : T(s) — ps + 
Besides, T should evolve according to the motion of the underlying 
snake. To account for these two goals, we follow the approach of 
Kalnins et al., a linear least squares fit of T, based on the parame¬ 
ter values from the previous frame propagated to the brush path of 
this frame (via the associated snake). However, our approach of¬ 
fers an extra degree of freedom: SLAM textures allow us to handle 
gracefully variations in phase and slope while preventing sliding ar¬ 
tifacts, especially when zooming. Also note that our method does 
not need a fitting algorithm robust to outliers, such as RANSAC used 
by Benard et al. [2010], because we are not mixing the parameteri- 
zations of multiple brush paths. 

Topological events. Brush paths rely on the connectivity of their 
associated snakes. When a snake is updated so are its brush paths. 
For example, snake resampling (Section 4.2) causes resampling of 
the paths. When a snake is split (Section 5.1), the brush paths cov¬ 
ering the break are also split; likewise for trimming, extension and 
deletion. However, when two snakes merge, their brush paths are 
not merged automatically, as their endpoints may differ in position 
and parameterization. Instead, a new merging operator ensures that 
the paths are continuous spatially and in parameter value. If not, 
both paths are kept (Figure 3). Otherwise, the two brush paths are 
joined and their mutual parameterization becomes: 

T(s) = ps + fi where p — ----- 

h + h 




Figure 7: Stylization examples, from left to right: monkey with circular arcs (above) and fuzzy SLAM texture over toon shading (below); knot 
with construction lines (above) and overdrawn wiggles (below); woman in two poses and three styles: arcs, loopy offsets, and overdrawn. 



To avoid brush path fragmentation over time, we also propose a 
leveling mechanism, similar in spirit to the “healing” process of 
Kalnins et al. [2003]. It progressively pushes the parameteriza¬ 
tion of two reasonably close brush paths to their common mean 
T, encouraging a fusion in subsequent frames. Note that we can 
explicitly deal with the periodicity of the parameterization during 
this process, which increases its efficiency. 

7 Results 

Snakes and their accompanying brush paths provide a robust, tem¬ 
porally coherent and evenly parameterized set of curves in screen 
space. Thanks to these qualities, we can explore styles that pre¬ 
vious approaches were unable to apply on animated 3D scenes of 
moderate complexity (Figure 7 and Figure 11). 

7.1 Style examples 

We can produce standard drawing styles by using SLAM textures 
together with stroke attributes such as color and thickness. Taper¬ 
ing [Northrup and Markosian 2000] is provided and can be com¬ 
puted per brush path or per snake. 

Offsets We can also apply displacement mapping to the vertices 
of each brush path, exploiting the fact that brush path geometry 
is persistent across frames. This mechanism supports, for example, 
the loopy effect added to the knot or the woman in Figure 7. Kalnins 
et al. [2002] synthesize screen space offsets using Markov random 
fields and apply them on the brush paths geometry. In contrast, we 
synthesize a texture of offsets using the SLAM synthesis approach 
of Benard et al. [2010] which provides similar details at varying 
scales. 

Overdraw The unit coverage computation at the snakes level not 
only prevents line clutter but also supports a form of “overdrawing,” 
i.e. the overlap of a controlled number of brush paths on the same 


snake. The user specifies a target overdraw G M > 1. We 
measure the current overdraw £2 C between two snake vertices as the 
number of brush paths overlapping this edge. For every brush path 
endpoint, we stochastically decide whether to extend, trim, or leave 
it unchanged, based on the probabilities: 

P (extend) = max(0, 7 ex tend(l - n c /fi t )) 

P(trim) max(0,7trim(l - Ll t /Q c )) 

where 7 ex tend and 7 tr im weight how rapidly the overdraw moves to¬ 
wards its target during animation. We can create even sketchier 
styles by combining overdraw with a small target length and ran¬ 
dom offsets per brush paths. 

Speed lines Brush paths are persistent across frames and have 
their own linear parameterization. We take advantage of these two 
properties to produce speed lines (Figure 8) inspired by lines ap¬ 
pearing in hand drawn comics. 

Speed lines are created as new brush paths formed by chaining to¬ 
gether copies of seed vertices over a series of frames. These seeds 
are chosen at locations undergoing noticeable motion, i.e. regions 
of brush paths where the velocity 'v has a strong component oppo¬ 
site to the projected surface normal. At every brush path vertex we 
compute V = —If • h. We also compute the mean and standard 
deviation cr v of V over all the vertices. A vertex is defined as a seed 
when V > t cr v + with t a threshold that controls how much of 
the model receives these effects (we use \). 

Speed lines can then be rendered as any other brush path using the 
effects described earlier. We generally find that it helps to trim the 
end of the speed lines after a given length. 




Figure 8: Speed lines stylized with arrows and dots textures. 












Figure 9: The Jot system of Kalnins et al. [2003] (a) generates 
many short strokes because the visible silhouettes produce short 
brush paths. Our approach (b) recovers connectivity using snakes, 
leading to fewer, longer strokes. 


7.2 Performance 

Our implementation is primarily CPU-based, except for the 2D line 
extraction, blur for relaxation, and final rendering which rely on the 
GPU. The system performs at interactive rates on a recent machine 
(Intel Core 2 at 2.40 Ghz with 4 GB of RAM and a GeForce GTX 
260) for models of moderate complexity (ranging from 20 to 30 
FPS on the models shown in this paper). A major bottleneck is 
the readback from the GPU to CPU, which is performed at several 
stages. Techniques such as stream compaction [Horn 2005] could 
be used to reduce this cost. The sparse matrix inversion required for 
the relaxation step is performed efficiently using the CHOLMOD 
library [Chen et al. 2008]. 

7.3 Comparison with Previous Approaches 

Our method is the first to allow a coherent parameterization of 
image-space feature lines. Implicit Brushes [Vergne et al. 2011] 
was the best available image space approach for line stylization. 
We obtain more complex styles with parameterization based effects 
(regular textures, tappering) and shape abstraction. 

Previous object-space methods [Kalnins et al. 2003; Benard et al. 
2010; Buchholz et al. 2011; Karsch and Hart 2011] rely on visible 
stretches of object space feature lines as input, and their brush paths 
can not be longer than these features. The impact of brush path 
length on the final rendering can be seen in Figure 9. The short, 
unpredictable paths of the silhouette segments are clearly visible, 
while the snakes provide long brush paths that correspond with the 
major shape features of the model. 

An alternative solution to our snake based approach could be to 
chain the small pieces of line together with an involved chaining 
algorithm (e. g., [Grabli et al. 2010]) before computing coherence 
using Kalnins et al. [2003] or Buchholz et al. [2011]. However, 
achieving interactive performances is very unlikely with such an 
approach. Moreover, vectorizing fully disconnected image space 
samples wouldn’t be possible. 

As a nice side effect, working in image space allows us to provide a 
LoD mechanism. While zooming in and out, the constant coverage 
radius ensures a unit line density for a given range of orientations 
(Figure 10). That way, brush paths appear and disappear naturally 
according to the camera motion as can be seen in the accompanying 
video. 




Figure 10: Zoom out on an octopus depicted with object space lines 
(left) and active strokes (right). The line screen space density varies 
with our approach avoiding line clutters (bottom). 

7.4 Limitations 

Our method deals with image-space lines. We therefore have little 
knowledge of the 3D informations of the scene. It mostly has an 
impact on occlusions and disocclusions. For these visibility events 
a snake does not know what are the best feature samples to follow. It 
creates a tearing effect where during 2 or 3 frames the snake zigzags 
between two feature lines. This is the price to pay to have an image- 
based approach allowing for efficient frame-rates that would not 
have been possible to obtain with an object-space approach. 

Another limitation of our approach is that for efficiency, decisions 
about snakes and brush paths are made locally. There is no ex¬ 
plicit mid- or high-level understanding of the scene. For example, 
tracking T-junctions might be a way to improve the behavior at oc¬ 
clusions. As a result, our approach is well-suited for feature lines 
that are view-dependent and extracted from organic shapes, rather 
than fixed features such as those found in architectural scenes. 

8 Conclusions and future work 

This paper describes an approach for coherent evolution of feature 
line rendering for animated 3D scenes. Feature lines are approx¬ 
imated by snakes that advect with the motion of the scene. The 
snakes adjust their arrangement and topology using a new set of 
operators, in order to match the changing scene. Attached to the 
snakes, brush paths carry parameterization from one frame to the 
next with coherence, and can therefore be stylized and abstracted 
with temporal coherence. We introduce several new stylization ef¬ 
fects shown in Section 7 and in the accompanying video. 

Our pipeline is generic enough to be applied on video, by using 
vision algorithms to extract an approximated version of the data 
that we need: optical flow, feature lines, and depth. This endeavor 
would be particularly timely with the advent of commodity cameras 
that supply depth and optical flow along with color information. 

Finally, while this system cannot optimize globally over an entire 
animation sequence (since we assume the frames are not known in 
advance) it might be possible to anticipate one or two frames in the 
future using something like a Kalman filter on the camera path and 
then use this information to broaden the scope of the optimization. 
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Figure 11: Animated models stylized with active strokes: running elephant rendered with circular acrs (left) and segments (middle); animated 
hand in three poses with SLAM texture and offsets. 


A Line Extraction with Steerable Filters 

Freeman and Adelson [1991] describe an image contour extractor 
using steerable quadrature pair filters. The quadrature pair responds 
equally to step-edges and fine lines in the image, providing a con¬ 
sistent, intuitively appealing set of line samples. The steerable for¬ 
mulation provides smooth, precise estimation of orientation, which 
we use for the tangent at each contour pixel. 

We apply the contour extraction to a shaded image, allowing us 
to capture lighting as well as shape effects (similar to [Lee et al. 
2007]). The image is optionally augmented with depth information 
to ensure that depth discontinuities are captured by lines. The input 
to the method is a scalar image I(x , y ) = L(x , y ) + /3z where x 
and y are the screen space pixel coordinates, L(x,y) is any shaded 
image, and (3 is a parameter that controls the influence of depth 
z. We apply to I the second-order quadrature filter pair contour 
detector of Freeman and Adelson [1991]. Finally, we perform non¬ 
maxima suppression using the estimated orientation to find the set 
of line samples. The parameters of the method are the filter kernel 
size, a noise threshold, and f3. 

The filter pair can be computed very efficiently on the GPU using 
the steerable formulation with seven basis filters. Each basis fil¬ 
ter can be expressed as the composition of a Gaussian and a small 
derivative kernel, where the Gaussian can be computed with a sep¬ 
arable formulation. We use this method for all the results with the 
exception of Figure 11. 
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